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TITLE OF THE INVENTION 
METHOD AND APPARATUS FOR PROCESSING IMAGES 
5 BACKGROUND OF THE INVENTION 

Field of the Invention 

[0001] The present invention relates to measuring a 
similarity index between captured images. 

10 Description of the Related Art 

[0002] With the widespread use of digital cameras and 
digital video cameras, captured static images or moving 
images are treated as electronic data. Further, cameras 
with pan heads, typified by WebView™ products, that can 

15 control moving images from remote locations or under 

automatic control are used. 

[0003] Attribute information of captured images generally 
includes dates, times, shutter speeds, and exposures. In 
the case of controllable cameras, such information may 
20 include pan angles, tilt angles, angles of view (zoom 

angles), and even various control information or 
supplementary information (e.g. the temperature or weather 
conditions at capturing time) . 

[0004] Moreover, surveillance systems having the above- 
25 mentioned cameras are in use and may store captured images 
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for later use. Since an enormous number of captured Images 
Is generally stored, especially In surveillance systems, it 
is necessary to reduce the enormous number of such images 
down to essential images. Hitherto, image retrieval using 
5 attribute information, such as the capture date and time, 
has been performed. 

[0005] There are image retrieval techniques according to 
the similarity index between images, for example, disclosed 
in Japanese Patent Laid-Open Nos. 2000-235637 and 2000- 

10 235638 (hereinafter referred to as References 1 and 2, 

respectively) . Using the similarity index between images 
allows, for example, retrieval of an image that is similar 
to a particular image or detecting a scene change in a TV 
program to find a part where images vary in moving images. 

15 [0006] However, such retrieval has a problem in that a 

retrieved image from an enormous number of images is not 
always the exact image that meets the needs of searchers. 
In particular, measurement of the similarity index depending 
on captured images themselves is insufficient for accurate 

20 measurement since all capturing conditions are not 
considered. 

[0007] Capturing with a movable camera or the like may 
produce images having different angles of view according to 
control, so that the captured images may include images that 
25 happen to show similarity, such as images whose captured 
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objects are themselves different but have similar color or 
shape. Therefore, retrieval depending solely on the 
similarity index between images has problems. 
[0008] Further, capturing with a movable camera or the 
5 like may include a case where a captured object of interest, 
which is specified as a rectangular region in a selected 
image, does not exist in other images due to the different 
angles of view or a case where the captured object exists at 
different locations in the images. 

10 [0009] In view of these circumstances, it is an object of 

the present invention to provide a method for processing 
images that reduces an enormous number of images down to 
images suitable for measuring the similarity index and that 
measures the similarity index between the reduced number of 

15 images more effectively. 

SUMMARY OF THE INVENTION 

[0010] The present invention provides a method for 
20 processing images comprising the steps of obtaining image 
data units and attribute information thereof; selecting a 
first image data unit from the image data units as a 
reference for measuring a similarity index; comparing 
attribute information of the first image data unit with 
25 attribute information of the remaining image data units 
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Other than the first Image data unit to determine a second 
image data unit to be used for measuring the similarity 
index; and measuring the similarity index between the first 
image data unit and the second image data unit. 
5 [0011] Further objects, features and advantages of the 

present invention will become apparent from the following 
description of the preferred embodiments with reference to 
the attached drawings. 

10 BRIEF DESCRIPTION OF THE DRAWINGS 

[0012] Fig. 1 is a block diagram showing a system 
configuration of an apparatus for processing images 
according to a first embodiment of the present invention. 
15 [0013] Fig. 2 is a flowchart showing an operating 

procedure for displaying images on the basis of an image - 
displaying program stored in the apparatus for processing 
images of the first embodiment. 

[0014] Fig. 3 illustrates a typical display of image data 
20 units read in the apparatus for processing images of the 
first embodiment. 

[0015] Fig. 4 is a flowchart showing a procedure for 
measuring a similarity index on the basis of a similarity- 
measuring program stored in the apparatus for processing 
25 images of the first embodiment. 
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[0016] Fig. 5 schematically illustrates a typical image 
that is split into blocks for calculating a similarity 
distance in the first embodiment. 

[0017] Fig. 6 illustrates a typical display of image data 
5 units sorted according to the similarity indices in the 

first embodiment. 

[0018] Fig. 7 is a flowchart showing a procedure for 
measuring a similarity index on the basis of a similarity- 
measuring program stored in an apparatus for processing 

10 images of a second embodiment. 

[0019] Fig. 8 schematically illustrates a typical display 
of an image that is split into blocks for calculating a 
similarity distance in the second embodiment. 
[0020] Fig. 9 schematically illustrates an example 

15 showing a specified rectangular region in a selected image 

and the corresponding rectangular region in an image to be 
compared with in a third embodiment . 

[0021] Fig. 10 illustrates a typical measurement of a 
similarity index between the specified rectangular region in 
20 the selected image and the corresponding rectangular region 

in the image to be compared with in the third embodiment. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 



25 



First Embodiment 
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[0022] The first embodiment of the present invention will 
be described below in detail. 

[0023] Fig. 1 is a block diagram showing a system 
configuration of an apparatus for processing images 
5 according to the first embodiment of the present invention. 
As shown in Fig. 1, the apparatus for processing images 
according to this embodiment includes a memory (MEM) 101 
that stores programs necessary for booting up the system or 
data; a central processing unit (CPU) 102 that performs 

10 computations and controls processing; a display device 

(DISP) 103 that displays information such as letters, 
symbols/ or images; an auxiliary storage (HDD) 104 that 
stores data or information; a keyboard (KB) 105 with which a 
user types letters or the like; a pointing device (MOUSE) 

15 106 by which a user selects displayed letters or images or 

the like with an on-screen cursor or the like; and a bus 109 
that connects the above-mentioned components. 
[0024] In this embodiment, the memory 101 stores an 
image -displaying program 110 and a similarity-measuring 

20 program 120. The auxiliary storage 104 stores n+1 image 

data units iO to in. 

[0025] The apparatus for processing images according to 
this embodiment obtains image data units and attribute 
information thereof and then stores them in the auxiliary 
25 storage 104, as will be described below. When an image data 



unit (a first image data unit) is selected as a reference 
for measuring of a similarity index, attribute information 
of the selected image data unit is compared to that of other 
image data units stored in the auxiliary storage 104 to 
exclude an image data unit that is not to be used for 
measuring the similarity index. In other words, attribute 
information of the image data unit (the first image data 
unit) for measuring the similarity index is compared to that 
of the remaining image data units other than the image data 
unit (the first image data unit), and an image data unit 
that is used for measuring the similarity index (a second 
image data unit) is determined. Then, the image data unit 
(the first image data unit) is compared to the image data 
unit that has not been excluded (i.e. , the second image data 
unit that is to be used for measuring the similarity index) . 
[0026] The operation of the apparatus for processing 
images having the above-mentioned arrangement will now be 
described in detail. Fig. 2 is a flowchart showing an 
operating procedure for displaying images on the basis of 
the image -displaying program 110 stored in the apparatus for 
processing images of the first embodiment. 

[0027] Firstly, the image data units iO to in, which are 
stored in the auxiliary storage 104, are read (step S201). 
These image data units are, for example, known JPEG image 
data or EXIF- standard data. A header of each image data 



unit Includes attribute values of the Image. In this 
embodiment, such attribute values Include the capture date 
and time, the exposure, the shutter speed, the pan angle P, 
the tilt angle T, and the angle of view (zoom angle) Z In 
each Image data unit* 

[0028] Subsequently, the Image data units are displayed 
by neune, as shown In Fig. 3, after they are sorted, for 
example, by date and tlme(step S202 of Fig. 2). Fig. 3 
Illustrates a typical display of Image data units read In 
the apparatus for processing Images In the first embodiment. 
There may be cases where Images themselves are displayed or 
both the names and the Images are displayed. Then, 
processing waits until an event occurs (step S203). 
[0029] If an event occurs by, for example, clicking a 
certain Item such as the shutter speed designated as 301 In 
Fig. 3 (the YES path of step S204), the Image data units are 
sorted and displayed according to the selected Item (step 
S205), as performed in Windows™ Explorer. 
[0030] If no item is determined to be clicked at step 
S204 (the NO path), it is then determined whether any of the 
names of the image data units, designated as 302 in Fig. 3, 
is clicked or not (step S206). If a name of an image data 
unit is clicked (the YES path), the image data unit having 
the clicked name is determined to be selected for measuring 
the similarity indices to other image data units (step S207). 
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[0031] If no name of an Image data unit is determined to 
be clicked at step S206 (the NO path), it is then determined 
whether a similarity-measuring item 304 is clicked or not 
(step S208). The similarity-measuring item 304 appears in a 
5 menu displayed upon clicking a Menu button 303 shown in Fig. 

3. Then, it is determined whether any image data unit as a 
reference for measuring the similarity index is selected or 
not (step S209). If an image data unit is determined to be 
selected (the YES path), the similarity-measuring program 

10 120 is performed (step S210). Subsequently, the processing 

goes back to step S203 and waits until the next event occurs. 
[0032] If the similarity-measuring item 304 is not 
clicked at step S208 (the NO path), the image -displaying 
program 110 completes upon selecting Exit 305 (the YES path 

15 of step S211). In other cases (the NO path of step S211), 
the processing goes back to step S203 and waits until the 
next event occurs. 

[0033] The typical operation according to the similarity- 
measuring program 120 is as follows. Fig. 4 is a flowchart 

20 showing a procedure for measuring the similarity index on 

the basis of the similarity-measuring program 120 stored in 
the apparatus for processing images of the first embodiment. 
[0034] Specifically, the calculation of the similarity 
indices is performed between the image selected at step S207 

25 shown in Fig. 2 and other images stored in the auxiliary 
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storage 104 (image-group for comparison). This calculation 
of the similarity index between images may use the 
calculation of similarity distances disclosed in References 
1 and 2, but it is not restricted to this disclosed 
calculation algorithm. 

[0035] The calculation of similarity distances according 
to this embodiment will now be described below. 
[0036] Firstly, the selected image and the image -group 
for comparison are split into a plurality of blocks 
lengthwise and breadthwise, as shown in Fig. 5. Then, 
average values of R, G, and B channels in each block are 
calculated. Fig. 5 schematically illustrates a typical 
image that is split into blocks for calculating similarity 
distances in this embodiment. 

[0037] Then, differences between each of the average 
values of R, G, and B channels of the selected image and 
those of the image-group for comparison are squared on a 
block-to-block basis, and the sum of the squared differences 
is determined. This is defined as the similarity distance 
between images, representing the similarity index. The 
smaller the similarity distance between images, the more 
similar the images; the larger the similarity distance 
between images, the less similar the images. In brief, this 
distance represents "the similarity of color arrangement 
between images . " 
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[0038] The calculation of the similarity distance between 
images may be expressed as: 

where i is a block number in processing; k is the number of 
5 blocks into which the image is split; Pur is the average 

value of the R-channel of an ith block of a selected image; 
PiiG is the average value of the G-channel of the ith block 
of the selected image; Pub is the average value of the B- 
channel of the ith block of the selected image; P2iR is the 
10 average value of the R-channel of the ith block of an image 

for comparison; Fzig is the average value of the G-channel of 
the ith block of the image for comparison; and P2±b is the 
average value of the B- channel of the ith block of the image 
for comparison. 

15 [0039] The result of the calculation according to 

expression (1) represents information on the similarity 
index (or similarity distance), designated as D. If each of 
the average values of the R, G, and B channels is in the 
range, for example, 0 to 255, the information D is in the 

20 range 0 to 255^ X 3 X k = 195075k. 

[0040] The similarity index S of the compared image is 
defined in the range of 0 to 100, as expressed by the 
following expression (2): 
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S ^ (195075k - D) / 195075k * 100 • • • (2) 

[0041] If the similarity index S is 100 in this case, the 
compared image is identical to the selected image. The 
larger the value of the more similar the images. 
5 [0042] The similarity index S, which is the result of 

calculating the similarity index, is then stored. The 
similarity indices to other images for comparison are 
determined in the same way. 

[0043] In this embodiment, pan angles, tilt angles, and 
10 angles of view (zoom angles) (hereinafter collectively 
referred to as PTZ) of the attribute information are 
compared between the selected image and other images before 
calculating the similarity indices. Specifically, if a pan 
angle Px, a tilt angle Tx, and an angle of view (a zoom 
15 angle) Zx of the selected image is different from a pan 
angle Py, a tilt angle Ty, and an angle of view (a zoom 
angle) Zy of a compared image, respectively, it is 
determined that the compared image is not to be used for 
measuring the similarity index; thus the similarity index is 
20 not calculated. In this case, the similarity index of the 
compared image may be asstimed to be, for example, "-1". 
[0044] In this embodiment, a (second) image data unit 
(i.e., an image data unit that are to be used for measuring 
the similarity index) is not included in image data units 
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having pan angles, tilt angles, and zoom angles of a camera 
during capturing the Image data units, these angles being 
different from those Included In the attribute Information 
of a (first) Image data unit (I.e., an Image data unit as a 
5 reference for measuring the similarity Indices to other 
Image data units). 

[0045] Additionally, the apparatus for processing Images 
according to this embodiment, as described above, splits 
each of the first Image data unit and the second Image data 
10 unit Into a plurality of blocks, and measures the similarity 

Index between the first Image data unit and the second Image 
data unit on a block-to-block basis. 

[0046] The measured similarity Indices are displayed on a 
screen as shown In Fig. 3 (step S402). Fig. 6 Illustrates a 

15 typical display of Image data units sorted according to the 

similarity Indices In the first embodiment. Referring to 
Fig. 6, the Image data units are displayed In descending 
order from an Image data unit having the largest similarity 
Index (I.e. , the Image data that Is determined to be very 

20 similar) based on the measured similarity Indices 601. 

[0047] Since Image data units showing similarity Indices 
of '-1* are excluded from measuring. It Is possible to omit 
them from the screen display. This allows only Image data 
units that are used for measuring the similarity Indices to 

25 be displayed, resulting In more effective measuring of the 
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similarity indices. VOien the similarity indices to all 
image data units to be used for measuring are calculated and 
then the image data units with the calculated similarity 
indices are displayed as illustrated in Fig. 6, the 
5 similarity-measuring program 120 exits. The mathematical 

expressions given above are merely examples, and the methods 
for measuring the similarity indices in applications of the 
present invention are not restricted. 
[0048] The method for retrieving image data units 

10 described above allows extraction of only image data units 

whose capture conditions (the PTZ in this embodiment) are 
considered, while extraction of image data by conventional 
calculation of the similarity index between images may 
include undesired images. Therefore, even when an enormous 

15 number of image data is stored, such as when PTZ- 

controllable cameras are used for suarveillance, the use of 
the apparatus for processing images according to this 
embodiment enables a reduction in the number of images that 
are used for measuring the similarity indices accurately. 

20 Attribute information other than the PTZ is applicable, as 
in the case of the above embodiment. 



25 



Second Embodiment 

[0049] The apparatus for processing images according to 
the first embodiment excludes images having different angles 
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of view during capturing to reduce the number of image data 
units to be used for measuring the similarity Indices. In 
contrast to this, an apparatus for processing images 
according to the second embodiment further performs 
5 specifying a rectangular region having an object in which a 
searcher is interested when the searcher selects an image 
data unit for measuring the similarity indices, thereby 
reducing the number of image data units to be used for 
measuring the similarity indices. 
10 [0050] The apparatus for processing images of this 

embodiment has a system configuration that is similar to 
that of the first embodiment except that a similarity- 
measuring program stored in the memory 101 shown in Fig. 1 
is different. 

15 [0051] The operation of the apparatus for processing 

images according to this embodiment will now be described in 
detail. The operation of displaying images according to the 
image -displaying program 110 is similar to that of the first 
embodiment. Fig. 7 is a flowchart showing a procedure for 

20 measuring the similarity index on the basis of the 

similarity-measuring progreim stored in the apparatus for 
processing images of the second embodiment. Fig. 8 
schematically illustrates a typical display of an image that 
is split into blocks for calculating similarity distances in 

25 the second embodiment. 
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[0052] Firstly, an Image selected as a reference for 
measuring the similarity index is displayed (step S701 of 
Fig. 7). As shown in Fig. 8, this displayed image has 
partitioning lines used for splitting the image into a 
5 plurality of blocks lengthwise and breadthwise, as with Fig. 
5. At this time, a Start Comparison button 801 is disabled, 
for example, by being grayed out on the screen display. 
Then, at least one block in which the searcher is interested 
is clicked with the pointing device 106 or selected with 

10 rubber banding in Windows™ to specify at least one block 

(the rectangular region) that is used for measuring (step 
S702). The Start Comparison button 801 is enabled and 
available only after at least one block is specified. 
[0053] Then, it is determined whether the Start 

15 Comparison button 801 is pressed or not (step S703). If the 
Start Comparison button 801 is pressed (the YES path), the 
similarity indices are calculated between the selected image 
and other images (image-group for comparison), as with step 
S401 of Fig. 4 showing the similarity-measuring program 120 

20 of the first embodiment (step S704). The procedure of the 

similarity-measuring program of this embodiment is different 
from that of the first embodiments in that expression (1) 
applies only to at least the one block specified at step 
S702. 

25 [0054] As Shown in Fig. 6, image data units having the 
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Similarity indices 601 are displayed as in step S402 of Fig. 
4 (step S705). As with the first embodiment, image data 
units having different PTZ attribute from those of the 
selected image are determined to be excluded from image data 
5 units to be used for measuring, so that the similarity 

indices thereof are not calculated. The similarity indices 
of such image data units may be assumed to be '-1' so that 
such image data units are not displayed. Upon completion of 
calculating the similarity indices to other image data units 
10 and displaying the image data units with the similarity 

indices, the similarity-measuring program according to this 
embodiment exits* 

[0055] The apparatus for processing images according to 
this embodiment specifies a region for measuring the 

15 similarity index in a first image data unit (i.e., an image 

data unit for measuring the similarity indices to other 
image data units) and measures the similarity index between 
the specified region in the first image data unit and the 
corresponding region in a second image data unit (i.e., an 

20 image data unit that is to be used for measuring the 

similarity indices). 

[0056] Additionally, the apparatus for processing images 
according to this embodiment splits each of the first image 
data unit and the second image data unit into a plurality of 
25 blocks, and specifies at least one block to specify the 



- 18 - 



region for the similarity index in the first image data unit 
(the image data unit for measuring the similarity indices to 
other image data units). 

[0057] As described above, the similarity index is 
5 calculated only in the rectangular region of interest that 
is specified at step S702 in this embodiment. Therefore, 
the calculation is completed without being affected by 
unnecessary portions of the image data unit, resulting in a 
measurement of the similarity indices with higher 
10 reliability. In particular, the combined use of the PTZ and 

block- specifying for measuring the similarity indices in a 
movable and controllable camera achieves a greatly reduced 
number of image data units. 



15 Third Embodiment 

[0058] The apparatus for processing images according to 
the second embodiment specifies the rectangular region that 
is to be used for comparison and an image data unit having 
the same angle of view as the selected image data unit is 

20 used for comparison. In contrast, an apparatus for 

processing images according to the third embodiment performs 
calculations for measuring the similarity indices to regions, 
corresponding to the specified rectangular region in the 
selected image data unit, of image data units having the 

25 different angle of view as well as images having the same 
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angle of view. This Is because a specified rectangular 
region In an Image captured with a controllable camera Is 
relatively small for the angle of view used in capturing, 
and thus the corresponding region may be included in images 
5 with different angles of view. 

[0059] The apparatus for processing images of this 
embodiment has a system configuration that is similar to 
that of the second embodiment except that a similarity- 
measuring program stored in the memory 101 shown in Fig. 1 

10 is different. 

[0060] The operation of the apparatus for processing 
images according to this embodiment will now be described in 
detail. The operation of displaying images according to the 
image -displaying program 110 is similar to that of the first 

15 embodiment. At step S201 of Fig. 2 showing the procedure 
for displaying images, images captured with different pan, 
tilt, and zoom operations of the camera may be read in as 
images having different angles of view. Examples of such 
Images are shown as regions of angles of view 901, 903, or 

20 904 in Fig. 9, which will be described below. The 

similarity-measuring program of this embodiment is similar 
to that of the second embodiment except that a calculation 
for measuring the similarity indices between images 
described at step S704 of the flowchart of Fig. 7 is 

25 different. 
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[0061] Specifically, according to the similarity- 
measuring program of this embodiment, an image including a 
rectangular region corresponding to a specified rectangular 
region in the selected image for measuring the similarity 
5 index is to be comparison with, and the corresponding 

rectangular region is extracted and is compared with the 
specified rectangular region. Accordingly, an image having 
no corresponding rectangular region is excluded from the 
comparison, and it is assumed to be, for example, '-1" to be 

10 not displayed. 

[0062] Fig. 9 schematically illustrates an example 
showing the specified rectangular region in the selected 
image and the corresponding rectangular region in the image 
to be compared with in the third embodiment. Referring to 

15 Fig. 9, the region 901 represents the angle of view of the 

selected image, a region 902 represents the specified 
rectangular region, and the region 903 represents the angle 
of view of a candidate image for comparison. In this case, 
since the region 903 includes a region corresponding to the 

20 specified rectangular region 902, the corresponding region 

is to be compared with. This corresponding region is shown 
as a rectangular region 902' in Fig. 10, which will be 
described below. 

[0063] Referring to Fig. 9, the region 904 represents the 
25 angle of view of another candidate image for comparison. In 



- 21 - 



this case, since the region 904 does not Include a region 
corresponding to the specified rectangular region 902, this 
candidate image is excluded from the comparison* Such a 
determination of whether a region is used for comparison or 
5 not is achieved by, for example, applying the coordinate 

axes to the whole effective region captured under control of 
pan, tilt, and zoom operations of the camera. In other 
words, the angles of view of candidate images are expressed 
with the absolute coordinates so that the determination of 

10 whether the candidate images include the entire specified 

rectangular region 902 or not can be performed by comparing 
the coordinates of the Images. Specifically, where any 
point in the upper-left part of the screen display has 
coordinate (0, 0), the rectangular region 902 can be 

15 represented as (XI, Y1)-(X2, Y2) and the region 903 can be 

represented as (X3, Y3)-(X4, Y4). If X1>=X3, X2=<X4, Y1>=Y3, 
and Y2=<Y4, the region 903 is determined to be used for 
comparison. 

[0064] Fig. 10 Illustrates a typical measurement of the 
20 similarity index between the specified rectangular region in 

the selected image and the corresponding rectangular region 
in the image that is to be used for comparison in the third 
embodiment. Specifically, the similarity index is measured 
between an image 1001, which is extracted with the specified 
25 rectangular region 902 of the region 901 of the selected 



- 22 - 



Image, and the corresponding rectangular region in the image 
that is used for comparison. An image 1002 is a rectangular 
image prepared by, for example, a typical scaling of an 
image that is extracted with the region 902' so as to be 
5 used for measuring the similarity index to the image 1001. 
[0065] In the apparatus for processing images according 
to this embodiment, image data units are obtained and stored 
in the auxiliary storage. A first image data unit is 
selected as a reference for measuring a similarity index and 

10 a region for measuring the similarity index in the first 

image data unit is specified. A region in a second image 
data unit is extracted from the remaining image data units 
other than the first image data and the specified region in 
the first image data unit. The region in the second image 

15 data unit corresponds to the specified region. Then, the 

similarity index is measured between the specified region in 
the first image data unit and the extracted corresponding 
region in the second image data unit. 
[0066] As described above, comparison between the 

20 rectangular regions allows measurement of the similarity 

indices to images stored as different images, images 
captured at different locations, or images having different 
angles of view. 

[0067] As described above, the similarity index is 
25 calculated between the specified rectangular region in the 
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selected Image and the corresponding region In the Image 
that is used for comparison, allowing extraction of similar 
images to be restricted to a desired region in this 
embodiment . 

5 

Other Embodiments 

[0068] The present invention is applicable to a system 
consisting of a plurality of devices, such as a host 
computer, an interface device, a reader, or a printer, or to 
10 an apparatus consisting of a single device, such as a 

copying machine, or a facsimile machine. 

[0069] The object according to the present invention can 
be achieved by providing a system or an apparatus with a 
recording medium (or a storage medium) on which program code 

15 for software achieving functions of the above described 

embodiments is recorded, wherein a computer (a CPU or an 
MPU) reads the program code stored on the recording medium 
and executes it. In this case, the program code read from 
the recording medium itself carries out the functions of the 

20 above described embodiments, so that the recording medium 

recording the program code is included in the present 
invention. Further, it is apparent that the functions of 
the above described embodiments can be carried out not only 
where the computer executes the read program code, but also 

25 where an operating system (OS) running on the computer 
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executes a part of or all of actual processing on the basis 
of the program code instructions. 

[0070] Moreover, it is apparent that the functions of the 
above described embodiments can be carried out where the 
program code read from the recording medium is written in 
memory included in an extension card inserted into the 
computer or an extension unit connected to the computer, and 
then a CPU or the like included in the extension card or the 
extension unit executes a part of or all of actual 
processing on the basis of the program code instructions. 
[0071] When the present invention is applied to the above 
mentioned recording medium, the recording medium stores 
program code corresponding to the procedures described in 
the above embodiments. As described above, the present 
invention reduces the enormous number of images down to 
images suitable for measuring the similarity indices and 
measures the similarity indices between the selected images 
more suitably. 

[0072] While the present invention has been described 
with reference to what are presently considered to be the 
preferred embodiments, it is to be understood that the 
invention is not limited to the disclosed embodiments. On 
the contrary, the invention is intended to cover various 
modifications and equivalent arrangements included within 
the spirit and scope of the appended claims. The scope of 



- 25 - 



the following claims is to be accorded the broadest 
interpretation so as to encompass all such modifications and 
equivalent structures and functions. 



